Systems and methods for automated analysis of cells and tissues

ABSTRACT

Systems and methods for rapidly analyzing cell containing samples, for example to identify morphology or to localize and quantitate biomarkers are disclosed.

Under 35 USC §119(e)(1), this application claims the benefit of priorU.S. provisional application ______ filed Apr. 19, 2001, and U.S.provisional application 60/114,721, filed Oct. 31, 2001, the contents ofwhich are incorporated herein by reference.

This invention was made with government support under ES011571 andCA081333 awarded by National Institute of Health. The government hascertain rights in the invention.

1. BACKGROUND OF THE INVENTION

Tissue microarray technology offers the opportunity for high throughputanalysis of tissue samples (Konen, J. et al., Nat. Med. 4:844-7 (1998);Kallioniemi, O. P. et al., Hum. Mol. Genet. 10:657-62 (2001); Rimm, D.L. et al. Cancer J. 7:24-31 (2001)). For example, the ability to rapidlyperform large scale studies using tissue microarrays can providecritical information for identifying and validating drug targets/prognostic markers (e.g. estrogen receptor (ER) and HER2/neu) andcandidate therapeutics.

Automated quantitative analysis of tissue samples in microarrays,however, presents several challenges, including heterogeneity of tissuesections, subcellular localization of staining, and the presence ofbackground signals. For example, depending on the type of tumor ortissue section being analyzed, the area of interest may represent nearlythe entire sample, or only a small percentage. For instance, apancreatic carcinoma or lobular carcinoma of the breast with substantialdesmoplastic response may show stromal tissue representing a largepercentage of the total area. If the goal of the assay is to determineepithelial cell expression of a given marker, a protocol must be usedthat evaluates only that region. The protocol must not only be able toselect the region of interest but also normalize it, so that theexpression level read from any given area can be compared with that ofother areas. Subcellular localization presents similar challenges.Comparisons of nuclear or membranous staining, for example, are quitedifferent from those in total cytoplasmic staining.

Certain methods (including confocal and convolution/deconvolutionmicroscopy) have been used to quantify expression of proteins at thecellular (or sub-cellular) level within a single high power field(Robinson, J. P. Methods Cell. Biol. 63:89-106 (2001); Shaw, P.Histochem. J. 26:687-94 (1994)). However, these are computationallyintensive and laborious techniques, which operate on multiple serialimages. As a result, the current standard for analysis of tissuemicroarrays, like tissue sections, is conventional pathologist-basedanalysis and grading of the sample according to scale.

Most biomarkers exhibit a parametric (normal, “bell-shaped”)distribution, and consequently are best analyzed by a continuous scale(e.g., 0 to 1000). Unfortunately, manual observation tends to be nominal(e.g. 1+, 2+, 3+), primarily because the human eye in unable to reliablydistinguish subtle differences in staining intensity. Several methodshave been developed to translate nominal manual observations into acontinuous scale. Foremost among these is the H-score where the percentof positively stained cells (0 to 100) is multiplied by the stainingintensity (e.g. 0 to 3) to make a theoretically continuous scale (0 to300). However, the inability to detect subtle differences in stainingintensity, particularly at the low and high ends of the scale, as wellas the tendency to round scores (e.g., 50% at 3+ for a score of 150,versus 47% at 3+ for a score of 141), limits the effectiveness of theH-score.

Automated systems and methods for rapidly analyzing tissue, includingtissue microarrays, that permit the identification and localization ofidentified biomarkers within tissues and other cell containing samples,are needed.

2. SUMMARY OF THE INVENTION

In one aspect, the invention features systems and methods for rapidlyanalyzing cell containing samples to localize and quantitate particularbiomarkers within cells. In one embodiment, the method is implemented bya computer and superimposes an image of the biomarker against an imageof a user defined area within the cell to determine whether thebiomarker is within the user defined area.

In another aspect, the invention features an algorithm that facilitatesthe optical analysis of an array of biological samples, despite imageirregularities, distortions, varying topologies, and the absence of oneor more elements.

Analysis of patient samples according to the systems and processesdescribed herein can he useful diagnostically (e.g. to identify patientswho have a particular disease, have been exposed to a particular toxinor are responding well to a particular therapeutic or organ transplant)and prognostically (e.g. to identify patients who are likely to developa particular disease, respond well to a particular therapeutic or beaccepting of a particular organ transplant). As new and better markersof disease become identified in the post-genomic era, the instantdescribed processes, which not only quantitate the markers, but alsodetermine their relative location within a cell, will increase inapplicability.

Automated analysis of cell containing preparations, as described herein,can provide a rapid assessment of the prognostic benefit of biomarkers.In addition, these automated techniques can identify associations thatare typically not revealed using manual techniques. Also, automatedanalysis can better discern subtle differences in staining intensity,particularly at the upper and lower extremes. The ability to detect lowlevel expression and distinguish it from no expression can provideimportant prognostic information. Furthermore, analysis of thesub-cellular distribution of certain biomarkers may elucidate previouslyunrecognized associations with patient survival.

Other features, objects, and advantages of the invention will beapparent from the following figures, detailed description and claims.

3. DESCRIPTION OF THE FIGURES

FIG. 1(A-D) shows separate monochromatic images of a colon carcinomataken after staining with fluorescently-tagged markers and combined intoa single color image as follows: DAPI (to visualize nuclei, blue),anti-cytokeratin (to distinguish tumor from non-tumor elements, green),and anti-alpha-catenin (to visualize cell membranes, red).

FIG. 2(A-D) shows a regression comparison of automated andpathologist-based scoring of estrogen receptor levels.

FIG. 3 is a flowchart of a method for identifying and accounting for therelative location of spots within an array.

FIG. 4 is a flowchart of a process for localizing a signal (e.g. abiomarker) within a locale.

FIG. 5 shows a tissue microarray.

FIG. 6 shows an optical microscope station.

4. DETAILED DESCRIPTION

4.1 General

In general, described herein are a collection of techniques that can beused for rapid, to automated analysis of cell containing samples,including tissues and tissue microarrays. While these techniques buildon one another and are described as a cohesive process, each techniquehas wide applicability and may be used individually or in combinationsother than those described below.

In one embodiment is featured a technique to identify the location ofspots within an image. The technique, termed “spotfinder”, can flexiblyidentify such locations despite image irregularities, distortions,varying topologies, and the absence of one or more elements. Althoughthe process is described for locating the position of histospots andidentifying missing histospots within tissue microarray images, thetechnique has broader application. More specifically it can be used tolocate elements and identify missing elements in any collection ofelements. Moreover, the process can be used on arrays of virtually anydimension and comprising a variety of elements. The specimens are notlimited by size or shape, nor must they be regularly spaced.

In another embodiment is featured a technique that can be used alone orin conjunction with spotfinder to optically localize and quantitate abiomarker within a cell. Though an image of a cellular preparationtypically features two dimensions, cellular preparations feature depth.For example, one cellular feature may rest atop another. This overlapcan potentially confuse image analysis software. A technique describedherein, dubbed RESA (Rapid Exponential Subtraction Algorithm), canapproximate a three dimensional image by subtracting out-of-focus imageelements. Thus, the impact of background features on an image can bereduced, permitting better image analysis.

Another technique described herein, dubbed PLACE (Pixel Based LocaleAssignment Compartmentalization of Expression), distinguishes betweendifferent cellular characteristics. For example, the technique candetermine the location of subcellular compartments within individualcells. A computer implementing this technique can, for instance, measurethe relative intensities of images derived from compartment-specificstains on a pixel-by-pixel basis. The computer then determines forindividual pixels within an image, the likelihood that the pixel tocorresponds to a particular locale or user defined area within the cell.Such analysis permits the computer to assign signals to a sub-cellularcompartment with an established degree of accuracy (e.g., 95%). Thetechnique can co-localize signals associated with particular biomarkerswith images of defined locales within cells.

Use of these techniques can enhance both the speed and accuracy ofautomated microarray analysis. FIG. 1 shows separate monochromaticimages of a colon carcinoma taken after staining withfluorescently-tagged markers and combined into a single color image asfollows: DAPI (to visualize nuclei, blue), anti-cytokeratin (todistinguish tumor from non-tumor elements, green), andanti-alpha-catenin (to visualize cell membranes, red) (panel A). Notethe significant degree of overlap between the subcellular compartments.A monochromatic image of the biomarker β-catenin, is taken (panel B,inset) and the intensity of each pixel in the image redistributedaccording to the relative signal intensity of the various compartmentsin panel A (blue=nuclei, red=membrane, green=cytoplasm).

Although the β-catenin expression in this tumor is predominantlymembrane-associated, the significant overlap in compartments in panel Aincorrectly assigns a significant amount of the signal to the nucleus(magenta and blue pixels, panel B). To aid in the removal of overlappingsignals, the monochromatic image of each sub-cellular compartment isexponentially subtracted from an out-of-focus image. Panel C shows acomposite of the exponentially subtracted images of DAPI andanti-alpha-catenin (blue and red, respectively), shown on a mask derivedfrom the anti-cytokeratin mask (green pixels). Pixels with too muchoverlap between channels are negated (<5%), as are non-tumor areas, asdefined by a mask generated from the anti-cytokeratin image. In panel D,the signal intensity from an exponentially subtracted image of thebiomarker (β-catenin, inset) is then redistributed according to thecompartments defined in panel C. This results in more accurateassignment of the biomarker to the membrane compartment, which can haveimportant prognostic significance. Since membrane-associatedbeta-catenin stabilizes cadherin-mediated adhesion by facilitating thecytoskeletal attachment of adhesion complexes, while nuclear-associatedbeta-catenin acts as a transcription factor, which up-regulates severalgenes important in cell proliferation and invasion and is considered anoncogene in this capacity, expression of beta-catenin alone does notprovide prognostic information. However its localization in the nucleuscan be an important indicator of carcinogenesis.

1. Spot-Finder

In one embodiment, shown in FIG. 3, the computer removes B1 anyatypically sized spots from the image. Atypically sized spots mayinclude, for example, images of fused spots and/or debris. The computerperforms the process automatically, though in other embodiments it mayallow use of user input to facilitate the process.

The computer then creates or accesses an opaque virtual mask that is thesize and shape of a typical spot. Using the virtual mask, the computerscans B2 the image to determine B3 where the mask first covers an areawith the highest average pixel intensity. The computer monitors thetotal intensity of the image during the scan and, because the mask isopaque, identifies the position of the mask when the total imageintensity is minimized. The computer identifies this area as the firstspot and sets B4 the pixels within this area to have zero intensity. Thecomputer also sets additional pixels within a predefined area aroundthis area to have zero intensity. This helps to differentiate betweenoverlapping spots.

After identifying the first spot, the computer again scans B2 the imageusing the mask to find the next area with the highest average pixelintensity. When the next area is found, the computer identifies it asthe second spot and sets the pixels in and surrounding this area to havezero intensity. The computer repeats this process until it can no longerfind areas of the image with sufficient intensity to qualify as spots.

The computer then identifies B5 a reference point (e.g., the center) ineach spot, and draws a line connecting the reference point of each spotto each nearest neighboring spot reference point, above, below, to theleft, and to the right. If the computer cannot identify a nearestneighbor in any of these directions (i.e., the spot is on the edge ofthe array), the computer draws a line from the center of the spot to thenearest edge of the image.

2. RESA and PLACE

Once the location of an image area of interest is determined, an opticalmicroscope can obtain a high resolution image at an appropriatewavelength to identify cellular features of interest. These featuresinclude the biomarker, also referred to as the “signal”, the cells of tointerest within the tissue section (referred to as the “cell mask”), ora user defined location within the cell mask, also referred to as the“locale”. The signal, the cell mask, and the locale are referred to as“channels”.

Referring to FIG. 4, a process 50 determines the region of interest inthe images by developing a mask from the cell mask channel (step C1).Next, the process applies this mask to the locale and signal channels(step C2). The process then removes out-of-focus information from themasked images, for example, in a process of pseudo-deconvolution (stepC3). Next, in a “pixel assignment” phase, the process identifiessubcellular features in the image, assigning pixels in the image to thelocales (step C4). Once the pixels are assigned, the computer maps thelocales onto the signal image (step C5), and quantifies the amount ofbiomarker in each locale. This phase is referred to as “signalassignment”. These steps are described in greater detail below.

Masking

During this process, the software identifies a region of interest in theimage of the stained cells of interest (i.e., the cell mask channel).The software masks the locale and signal channels avoiding unnecessaryanalyses of areas outside the region of interest.

To identify a region of interest, the computer determines a thresholdintensity for the cell mask channel. Once determined, the computerredistributes the pixel intensities in a binary redistribution. In otherwords, the computer sets the intensity of each pixel below the thresholdto zero, and sets the remaining pixels to have the maximum intensity(e.g., for an 8-bit image the maximum intensity is 255). The set ofpixel locations set to maximum intensity are referred to as the mask.Subsequent procedures on the other images in the image stack areperformed on the pixel locations corresponding to the mask.

The threshold intensity is related to the intensity of the background inthe image, which the computer determines by first binning each pixelaccording to its intensity (e.g., in an 3-bit image each pixel will havean intensity from 0 to 255). In some embodiments, the backgroundcorresponds to the largest bin (i.e., the most common pixel intensity).In other embodiments, the background corresponds to the second largestbin. This occurs in some cases when the tissues autofluoresce and thelargest bin corresponds to an area of fluorescing tissue instead of thefluorescing histochemical stains. In either case, the computer assumesthat the background intensity is lower than a certain fraction of themaximum intensity (e.g., less than half the maximum intensity).

Bin size is plotted versus intensity to yield a histogram. The maximumpeak in the histogram corresponds to the largest bin. In embodimentswhere the largest bin corresponds to the background, the computerassigns the maximum peak intensity as the background intensity. In otherembodiments, where the background corresponds to the second largest bin,the histogram has a second peak at a lower intensity than the maximumpeak. So, if the second peak is at least a certain fraction of the sizeof the maximum peak (e.g., at least five percent), then the computerassigns the second peak intensity as the background intensity of theimage.

Once established, the computer adds an additional factor to thebackground intensity to determine the threshold intensity. For an 8-bitimage, this factor equals D (⅕)/10 multiplied by a user defined input(usually 0.5). Here, D(⅕) is the quintile distribution of the binnedpixels, which is determined as

D(⅕)=

I

_(top20) −

I

_(bottom 20),

where

I

_(top20) is the mean pixel intensity of the pixels within the top20^(th) percentile, and

I

_(bottom20) is the mean pixel intensity of pixels in the bottom 20^(th)percentile.

Pixels with intensity at or above the threshold intensity are assignedto the mask.

The mask is then further modified according to user-defined parametersand image processing techniques. For example, the mask can be dilated oreroded so that the mask area matches a user-defined value, or have holesof a particular (user-defined) size within it filled. The user-definedparameters for creating the mask may be adjusted after analyzing a smallnumber of sample histospot images, prior to running the entire array.

After developing the mask, the computer applies the mask to the imagesin the image slack, identifying the region of interest in each of theseimages as the pixel locations corresponding to the mask pixel locations.

Background Reduction

As shown, the process 50 reduces C3 the impact of the out-of-focusinformation from the image. For example, the process 50 may use a pseudodeconvolution technique. While the pixels of the remaining image arereduced in intensity, the image information represents a thinner virtualslice through the top of the tissue. Furthermore, pseudo-deconvolutionenhances the interfacial areas between the higher stain intensity andlower stain intensity areas of the image by increasing the contrastbetween these areas.

The computer performs pseudo-deconvolution on the locale (cellularcompartments) and signal (i.e., cellular components) channels. Thecomputer first masks the images of these channels, reducing the numberof pixels to be analyzed. The computer analyzes two images of eachchannel. The first image is an in-focus image (i.e., an image of the topof the histospot). The second image is a slightly out-of-focus image,produced by placing the focal plane slightly below the bottom of thetissue (e.g., for a five micron thick histospot, the focal plane of thisimage is located about eight microns below the top of the histospot).

For each pixel location, the computer subtracts a percentage of theout-of-focus image pixel intensity, I_(out-of-focus), from thecorresponding in-focus image pixel intensity. The computer determinesthe adjusted pixel intensity, I_(new pixel), using the quartiledistribution, D(¼), of the in-focus image as follows:

$\mspace{20mu} {{I_{{new}\mspace{14mu} {pixel}} = {I_{i\; n\text{-}\text{?}} - {I_{{out}\text{-}{of}\text{-}\text{?}} \times \left( \frac{I_{{ma}\; x} - I_{i\; {n \cdot \text{?}}}}{I_{{ma}\; x}} \right)^{\text{?}}}}},{\text{?}\text{indicates text missing or illegible when filed}}}$

where I_(max) is the maximum pixel intensity (e.g., 255 for an 8-bitimage), and ψ is calculated from

ψ=α×D(¼)^(−β),

which was developed from an empiric assessment of a library of images.Optical deconvolutions were judged visually and the ψ for each wasplotted versus the quartile distribution for the in-focus image.Regression analysis of the empiric data yielded values for thefitting-parameters (i.e., α is about 80 and β is about 1.19). Thequartile distribution is determined from

D(¼)=

I

_(top25) −

I

_(bottom25),

where

I

_(top25) is the mean pixel intensity of the pixels within the top25^(th) percentile, and

I

_(bottom25) is the mean pixel intensity of pixels in the bottom 25^(th)percentile. Conceptually, low intensity pixels in images with a low D(¼)(i.e. a low signal to noise ratio) are subtracted less heavily than lowintensity pixels from images with a high D(¼).

The value of ψ may be refined by determining the percent of signalintensity remaining after pseudo-deconvolution within the masked areaand comparing it to a predefined value for that channel. If the percentis, for example, greater than the predefined value then thepseudo-deconvolution stops. Otherwise, the computer iterativelyincreases the value of ψ until the predefined percent of signalintensity is reached. The predefined value is the expected percentage ofthe mask covered by a channel.

After pseudo-deconvolution, each pixel of the resulting images isassigned to a locale in a process referred to as pixel assignment.

Pixel Based Locale Assignment Compartmentalization of Expression (PLACE)

During the pixel assignment phase, the computer assigns an identitybased on the relative intensity of that pixel location in each of thelocale channel images (i.e., the images of the stained locales). Forexample, during this phase the computer decides for each pixel locationin the image whether it belongs to the nucleus, the membrane, or thecytoplasm. The computer does not make an assignment to pixels that itcannot assign within a user-defined degree of confidence (e.g., 95%).higher levels of confidence eliminate more pixels from the analysis.

In general, for each pixel location in two locale images the computerreads a pixel intensity and compares each intensity value to apredetermined threshold intensity value. If the intensity value of onlyone locale is greater than the threshold, the computer assigns the pixellocation to that locale. If both the intensity values are greater thantheir respective thresholds, the computer compares the intensity valuesfrom each locale, and assigns the identity of the locale having thegreater intensity to that pixel location. If both the pixel intensitiesare below their threshold values, the computer assigns the pixel to athird locale.

After repeating the above for pixel locations in the images, thecomputer calculates the area of each locale, and compares the result toa predetermined (expected) coverage fraction. If the calculated coveragefraction (e.g., number of nuclear locale pixels/number of masked pixels)is greater than the predetermined coverage fraction, then the computerremoves the pixels having the lowest intensity from the locale. Thecomputer continues to remove the lowest intensity pixels until thecoverage fraction is reduced to about the predetermined coveragefraction.

The following is an example of how this process works. The membranelocale and the nucleus locale images are selected for assignmentanalysis performed at 95% confidence interval. Pixel locations areassigned to the cytoplasm locale by exclusion.

The computer reads pixel intensities at each pixel location in themembrane and nucleus locale images, and compares them to thresholdvalues. If neither of the intensity values are greater than thethreshold values, the pixel location is assigned to the cytoplasmlocale. If only the nuclear intensity or membrane intensity is greaterthan the threshold value, the computer assigns the pixel location to theabove-threshold locale. If both intensities are higher than thethresholds, computer compares the ratio of the intensity values to one,and makes an assignment as follows:

${{\frac{{nuclear}\mspace{14mu} {intensity}}{{membrane}\mspace{14mu} {intensity}} > 1}->{{pixel}\mspace{14mu} {location}}} = {{nuclear}\mspace{14mu} {locale}}$${{\frac{{nuclear}\mspace{14mu} {intensity}}{{membrane}\mspace{14mu} {intensity}} < 1}->{{pixel}\mspace{14mu} {location}}} = {{{membrane}\mspace{14mu} {{locale}.\frac{{nuclear}\mspace{14mu} {intensity}}{{membrane}\mspace{14mu} {intensity}}}} = {{1->{{pixel}\mspace{14mu} {location}}} = {unassigned}}}$

Thus, if the nuclear intensity is greater than the membrane intensity,the computer assigns the pixel location to the nuclear locale. If themembrane intensity is greater than the nuclear intensity, the computerassigns the pixel location to the membrane locale. If the membraneintensity is equal to the nuclear intensity, the pixel location isunassigned. This repeats for the pixel locations.

Once all the pixel locations have been analyzed, the computer determinesthe amount of nuclear intensity incorrectly assigned to the membranelocale (i.e., nuclear to membrane spill-over), and vice versa. If theamount of nuclear intensity incorrectly assigned to the membrane tochannel is >5% of the total nuclear intensity, then the computer weightsthe nuclear intensity by a factor, w, and recalculates the ratio ofweighted nuclear intensity to membrane intensity. This ratio is comparedto one, and pixel locations are reassigned as follows:

${{\frac{w \times {nuclear}\mspace{14mu} {intensity}}{{membrane}\mspace{14mu} {intensity}} > 1}->{{pixel}\mspace{14mu} {location}}} = {{nuclear}\mspace{14mu} {locale}}$${{\frac{w \times {nuclear}\mspace{14mu} {intensity}}{{membrane}\mspace{14mu} {intensity}} < 1}->{{pixel}\mspace{14mu} {location}}} = {{{membrane}\mspace{14mu} {{locale}.\frac{w \times {nuclear}\mspace{14mu} {intensity}}{{membrane}\mspace{14mu} {intensity}}}} = {{1->{{pixel}\mspace{14mu} {location}}} = {unassigned}}}$

The computer again determines the amount of each locale incorrectlyassigned. If this is still >5% the computer increases the value of w andreiterates the steps above. This continues until the amount ofincorrectly assigned nuclear locale is <5%. The computer employs asimilar technique to minimize the membrane-to-nuclear spillover.

The computer also calculates the area of the cytoplasmic (exclusion)locale and compares it to a predetermined value. By iterating theassignment process, the computer ensures that there is <5%cytoplasmic-to-nuclear or cytoplasmic-to-membrane, as determined basedon the biology.

The computer then evaluates the amount of signal in each locale during a“signal assignment” process.

Following pixel assignment, the computer sums the signal in each locale.The computer reads the pixel intensity of the signal image (i.e., theimage of the stain that selectively labels the cellular component), andadds together the signal intensity for pixel locations assigned to likesubcellular compartments. The computer calculates a pixel intensity sumof a locale by the direct addition of the signal intensity of each pixellocation assigned to that locale. The computer also calculates a sum ofpixel intensity ratios by adding together the ratio of the signalintensity and the locale intensity for each pixel location.

The pixel intensity sum and pixel intensity ratio sum is then used incalculating one or more parameters. For example, the computer determinesthe relative percentage of signal falling within each of thecompartments (e.g. 30% of the total signal is membranous, 20% iscytoplasmic, and 50% is nuclear). In another example, the computerexpresses the amount of signal present relative to the size of aparticular compartment (e.g. the signal intensity of pixels assigned tothe membrane channel divided by the number of pixels assigned to themembrane channel). The user may select to have the computer evaluateother parameters of interest. For example, how much of the image area iscovered by the mask, how much of the mask is covered by each locale,etc.

By implementing the pseudo-deconvolution algorithm (which limits themajority of extraneous pixel intensity) together with intensity areameasurements (which further define the area of a particular sub-cellularlocale), the computer is able to make highly accurate assignments ofpixel locations sub-cellular locations.

In some embodiments, the computer performs additional steps to betterutilize the dynamic range of the camera. This is achieved byredistributing the pixel intensities in an image across the dynamicrange of the detector based on their relative intensities.

One form of redistribution is normalized redistribution, whereby thelower threshold (i.e., the pixel intensity of the background, determinedduring masking) is subtracted from all the pixels in the image, and anypixel with a resulting negative value is set to zero. Normalizedredistribution is used for the signal channel as this redistributionpreserves the scale from one sample to the next, allowing directcomparisons to be made between samples. This is performed either aftermasking the signal image.

Double-logarithmic redistribution sets all pixels in an image above 50%of the image's upper threshold (i.e., the value which only 50% of thepixels in the image have greater intensity) to the maximum intensityvalue (e.g., 255 for an 8-bit image). All pixels with intensity valuesbelow the lower threshold are set to 0, and all pixels with intensityvalues between the upper and lower thresholds are reassigned accordingto the formula:

$\mspace{20mu} {I_{new} = {{\frac{\log \left( {I_{old} - {LT}} \right)}{I_{\text{?}}{\log \left( {{\frac{1}{2}{UT}} - {LT}} \right)}}.\text{?}}\text{indicates text missing or illegible when filed}}}$

where I_(new) refers to the new pixel intensity, I_(old) refers to theold pixel intensity, LT and UT are the lower and 50% maximum thresholds,respectively, and I_(max) is the maximum intensity value.Double-logarithmic redistribution is used for the locale channels,either after masking or after pseudo-deconvolution of these channels.Conceptually, it ensures that pixels in locale images that haveintensities above the 50^(th) percentile are assigned to their localeduring the assignment phase. Pixels with intensities below, but closeto, the 50^(th) percentile are weighted more heavily and are more likelyto be assigned to the locale than pixels that have intensities wellbelow the 50^(th) percentile.

Other user-defined redistributions, such as linear redistributions orother equation-based redistributions, may be used in addition to theabove-described examples.

Although the algorithms described above are with reference to analysisof tissue microarrays, they are not limited to studying only sucharrays. The spotfinder algorithm may be used for identifying thelocation of any element comprising a collection and the RESA and PLACEalgorithms may be used to localize and quantitate a biomarker within anyimageable, cell containing sample, including tissue biopsies and cellcontaining fluid samples, such as blood, urine, spinal fluid, saliva,lymph, pleural fluid, peritoneal fluid and pericardial fluid.

Also, any optical or non-optical imaging device can be used, such as forexample, upright or inverted optical microscopes, scanning confocalmicroscopes, cameras, scanning or tunneling electron microscopes,scanning probe microscopes, and imaging infrared detectors etc.

In the embodiments described above, the computer can include hardware,software, or a combination of both to control the other components ofthe system and to analyze the images to extract the desired informationabout the histospots and tissue microarrays. The analysis describedabove is implemented in computer programs using standard programmingtechniques. Such programs are designed to execute on programmablecomputers each comprising a processor, a data storage system (includingmemory and/or storage elements), at least one input device, at least oneoutput device, such as a display or printer. The program code is appliedto input data (e.g., stitched together images or image stacks) toperform the functions described herein and generate information (e.g.,localization of signal), which is applied to one or more output devices.Each computer program can be implemented in a high-level procedural orobject-oriented programming language, or an assembly or machinelanguage. Each such computer program can be stored on a computerreadable storage medium (e.g., CD ROM or magnetic diskette) that whenread by a computer can cause the processor in the computer to performthe analysis described herein.

The following provides a detailed description of a specific embodimentof the preparation and analysis of tissue microarrays according tomethods described herein, although similar steps could be performed withrespect to any cell containing sample. Referring to FIG. 5, a tissuemicroarray 100 includes multiple samples of histospots 120 prepared fromhistocores embedded typically in a thin (e.g., about five microns) blockof paraffin 130 at regular intervals, forming a series of rows andcolumns. Histospots (thin sections of histocores) 120 may besubstantially disk-like in shape and will typically have the samethickness as the paraffin block 130 (i.e., about five microns) and adiameter of about 0.6 millimeters. Typically the centers of thehistospots are spaced about a few tenths of a millimeter apart. Paraffinblock 130 and histospots 120 may be mounted on a microscope slide 110. Atissue microarray 100 may include any number of histospots, typically onthe order of several hundred to a few thousand.

Referring to FIG. 6, an optical microscopy station can be used to obtainan appropriate image of the tissue. Microscopy station 200 includes aninverted optical microscope 201 for imaging the tissue, and a computer290 for analyzing the images. Optical microscope 201 includes a mount210, housing a light source 220, a sample stage 240, an objective lens250 and a CCD camera 270. A frame grabber in computer 290 acquires theimages through CCD camera 270.

Optical microscope 201 also includes filter wheels 230 and 260, whichhouse a series of dichroic filters. The filters in wheel 230 allowselection of the appropriate illumination spectra for standard orfluorescent microscopy. Filters in wheel 260 filter the transmittedlight for isolation of spectral signatures in fluorescent microscopy.Sample stage 240 supports and appropriately positions tissue microarray100. Sample stage 240 can be linearly translated in the x, y, and zdirections (axes are shown). Sample stage 240 includes motors to enableautomated translation. Computer 290 controls sample stage 240translation by servo control of the motors.

A tissue microarray 100 can be imaged as follows: a user places themicroarray on a sample stage 240. The user adjusts sample stage 240 sothat the first (i.e., top-left) histospot is at the center of the fieldof view and focused on CCD camera 270. The objective lens 250 should beadjusted to the appropriate resolution, for example, a 0.6 millimeterhistospot can be viewed at 10× magnification. Generally, the histospotscorrespond to areas of higher light intensity than the surroundingparaffin, as assessed through various means including signals derivedfrom the visible light scattering of stained tissues, tissueautofluorescence or from a fluorescent tag. Computer 290 can acquire alow-resolution image (e.g. 64 pixel×64 pixel with 16 bin resolution)using computer software (Softworx 2.5, Applied Precision, Issaquah,Wash.) and an imaging platform (e.g., Deltavision). Computer 290automatically translates sample stage 240 by an amount approximatelyequal to a field of view. The computer then acquires a secondlow-resolution image. This process is repeated until the computer hasacquired images of the entire tissue microarray. Then, usingcommercially available software, the computer generates a compositeimage of the entire tissue microarray by stitching together the sequenceof images like a patchwork.

Biological markers, which may be detected in accordance with the presentinvention include, but are not limited to any nucleic acids, proteins,peptides, lipids, carbohydrates or other components of a cell. Certainmarkers are characteristic of particular cells, while other markers tohave been identified as being associated with a particular disease orcondition. Examples of known prognostic markers include enzymaticmarkers such as galactosyl transferase II, neuron specific enolase,proton ATPase-2, and acid phosphatase. Hormone or hormone receptormarkers include human chorionic gonadotropin (HCG), adrenocorticotropichormone, carcinoembryonic antigen (CEA), prostate-specific antigen(PSA), estrogen receptor, progesterone receptor, androgen receptor,gClq-R/p33 complement receptor, IL-2 receptor, p75 neurotrophinreceptor, PTH receptor, thyroid hormone receptor, and insulin receptor.

Lymphoid markers include alpha-1-antichymotrypsin, alpha-1-antitrypsin,B cell marker, bcl-2, bcl-6, B lymphocyte antigen 36 kD, BM1 (myeloidmarker), BM2 (myeloid marker), galectin-3, granzyme B, HLA class IAntigen, HLA class II (DP) antigen, HLA class II (DQ) antigen, HLA classII (DR) antigen, human neutrophil defensins, immunoglobulin A,immunoglobulin D, immunoglobulin G. immunoglobulin M, kappa light chain,kappa light chain, lambda light chain, lymphocyte/histocyte antigen,macrophage marker, muramidase (lysozyme), p80 anaplastic lymphomakinase, plasma cell marker, secretory leukocyte protease inhibitor, Tcell antigen receptor (JOVI 1), T cell antigen receptor (JOVI 3),terminal deoxynucleotidyl transferase, unclustered B cell marker.

Tumour markers include alpha fetoprotein, apolipoprotein D, BAG-1 (RAP46protein), CA19-9 (sialyl lewisa), CA50 (carcinoma associated mucinantigen), CA125 (ovarian cancer antigen), CA242 (tumour associated mucinantigen), chromogranin A, clusterin (apolipoprotein J), epithelialmembrane antigen, epithelial-related antigen, epithelial specificantigen, gross cystic disease fluid protein-15, hepatocyte specificantigen, heregulin, human gastric mucin, human milk fat globule, MAGE-1,matrix metalloproteinases, mclan A, melanoma marker (HMB45), mesothelin,metallothionein, microphthalmia transcription factor (MITF), Muc-1 coreglycoprotein. Muc-1 glycoprotein, Muc-2 glycoprotein, Muc-5ACglycoprotein, Muc-6 glycoprotein, myeloperoxidase, Myf-3(Rhabdomyosarcoma marker), Myf-4 (Rhabdomyosarcoma marker), MyoD1(Rhabdomyosarcoma marker), myoglobin, nm23 protein, placental alkalinephosphatase, prealbumin, prostate specific antigen, prostatic acidphosphadase, prostatic inhibin peptide, PTEN, renal cell carcinomamarker, small intestinal mucinous antigen, tetranectin, thyroidtranscription factor-1, tissue inhibitor of matrix metalloproteinase 1,tissue inhibitor of matrix metalloproteinase 2, tyrosinase,tyrosinase-related protein-1, villin, von Willebrand factor.

Cell cycle associated markers include apoptosis protease activatingfactor-1, bcl-w, bcl-x, bromodeoxyuridine, CAK (cdk-activating kinase),cellular apoptosis susceptibility protein (CAS), caspase 2, caspase 8,CPP32 (caspase-3), CPP32 (caspase-3), cyclin dependent kinases, cyclinA, cyclin B1, cyclin D1, cyclin D2, cyclin D3, cyclin E, cyclin G, DNAfragmentation factor (N-terminus), Fas (CD95), Fas-associated deathdomain protein, Fas ligand, Fen-1, IPO-38, Mcl-1, minichromosomemaintenance proteins, mismatch repair protein (MSH2), poly (ADP-Ribose)polymcrase, proliferating cell nuclear antigen, p16 protein, p27protein, p34cdc2, p57 protein (Kip2), p105 protein, Stat 1 alpha,topoisomerase I, topoisomerase II alpha, topoisomerase III alpha,topoisomerase II beta.

Neural tissue and tumour markers include alpha B crystallin,alpha-intemexin, alpha synuclein, amyloid precursor protein, betaamyloid, calbindin, choline acetyltransferase, excitatory amino acidtransporter 1, GAP43, glial fibrillary acidic protein, glutamatereceptor 2, myelin basic protein, nerve growth factor receptor (gp75),neuroblastoma marker, neurofilament 68 kD, neurofilament 160 W,neurofilament 200 kD, neuron specific enolase, nicotinic acetylcholinereceptor alpha4, nicotinic acetylcholine receptor beta2, peripherin,protein gene product 9, S-100 protein, scrotonin, SNAP-25, synapsin I,synaptophysin, tau, tryptophan hydroxylase, tyrosine hydroxylase,ubiquitin.

Cluster differentiation markers include CD1a, CD1b, CD1c, CD1d, CD1e,CD2, CD3delta, CD3epsilon, CD3gamma, CD4, CD5, CD6, CD7, CD8alpha,CD8beta, CD9, CD10, CD11a, CD11b, CD11c, CDw12, CD13, CD14, CD15, CD15s,CD16a, CD16b, CDw17, CD18, CD19, CD20, CD21,CD22, CD23, CD24, CD25,CD26, CD27, CD28, CD29, CD30, CD31, CD32, CD33, CD34, CD35, CD36, CD37,CD38, CD39, CD40, CD41, CD42a, CD42b, CD42c, CD42d, CD43, CD44, CD44R,CD45, CD46, CD47, CD48, CD49a, CD49b, CD49c, CD49d, CD49e, CD49f, CD50,CD51, CD52, CD53, CD54, CD55, CD56, CD57, CD58, CD59, CDw60, CD61,CD62E, CD62L, CD62P, CD63, CD64, CD65, CD65s, CD66a, CD66b, CD66c,CD66d, CD66e, CD66f, CD68, CD69, CD70, CD71, CD72, CD73, CD74, CDw75,CDw76, CD77, CD79a, CD79b, CD80, CD81, CD82, CD83, CD84, CD85, CD86,CD87, CD88, CD89, CD90, CD91, CDw92, CDw93, CD94, CD95, CD96, CD97,CD98, CD99, CD100, CD101, CD102, CD103, CD104, CD105, CD106, CD107a,CD107b, CDw108, CD109, CD114, CD115, CD116, CD117, CDw119, CD120a,CD120b, CD121a, CDw121b, CD122, CD123, CD124, CDw125, CD126, CD127,CDw128a, CDw128b, CD130, CDw131, CD132, CD134, CD135, CDw136,CDw137,CD138, CD139, CD140a, CD140b, CD141, CD142, CD143, CD144, CDw145,CD146, CD147, CD148. CDw149,CDw150, CD151, CD152, CD153, CD154, CD155,CD156, CD157, CD158a, CD158b, CD161, CD162, CD163, CD164, CD165, CD166,and TCR-zeta.

Other cellular markers include centromere protein-F (CENP-F), giantin,involucrin, lamin A&C [XB10], LAP-70, mucin, nuclear pore complexproteins, p180 lamellar body protein, ran, r, cathepsin D, Ps2 protein,Her2-neu, P53, S100, epithelial marker antigen (EMA), TdT, MB2, MB3,PCNA, and Ki67.

Cell containing samples may be stained using dyes or stains, orhistochemicals, that directly react with the specific biomarkers or withvarious types of cells or subcellular compartments. Not all stains arecompatible. Therefore the type of stains employed and their sequence ofapplication should be well considered, but can be readily determined byone of skill in the art. Such histochemicals may be chromophoresdetectable by transmittance microscopy or fluorophores detectable byfluorescence microscopy. In general, a cell containing samples may beincubated with a solution comprising at least one histochemical, whichwill directly react with or bind to chemical groups of the target. Somehistochemicals must be co-incubated with a mordant, or metal, in orderto allow staining. A cell containing sample may be incubated with amixture of at least one histochemical that stains a component ofinterest and another histochemical that acts as a counterstain and bindsa region outside the component of interest. Alternatively, mixtures ofmultiple probes may be used in the staining, and provide a way toidentify the positions of specific probes.

The following, non-limiting list provides exemplary chromophores thatmay be used as histological stains or counterstains and their targetcells, subcellular compartments, or cellular components: Eosin (alkalinecellular components, cytoplasm), Hematoxylin (nucleic acids), Orange G(red blood, pancreas, and pituitary cells), Light Green SF (collagen),Romanowsky-Giemsa (overall cell morphology), May-Grunwald (blood cells),Blue Counterstain (Trevigen), Ethyl Green (CAS) (amyloid),Feulgen-Naphthol Yellow S (DNA), Giemsa (differentially stains variouscellular compartments), Methyl Green (amyloid), pyronin (nucleic acids),Naphthol-Yellow (red blood cells), Neutral Red (nuclei), Papanicolaoustain (which typically includes a mixture of Hematoxylin, Eosin Y,Orange G and Bismarck Brown mixture (overall cell morphology), RedCounterstain B (Trevigen), Red Counterstain C (Trevigen), Sirius Red(amyloid), Feulgen reagent (pararosanilin) (DNA), Gallocyanin chrom-alum(DNA), Gallocyanin chrom-alum and Naphthol Yellow S (DNA), MethylGreen-Pyronin Y (DNA), Thionin-Feulgen reagent (DNA), Acridine Orange(DNA), Methylene Blue (RNA and DNA), Toluidine Blue (RNA and DNA),Alcian blue (carbohydrates), Ruthenium Red (carbohydrates), Sudan Black(lipids), Sudan IV (lipids), Oil Red-O (lipids), Van Gieson's trichromestain (acid fuchsin and picric acid mixture) (muscle cells), Massontrichrome stain (hematoxylin, acid fuchsin, and Light Green mixture)(stains collagen, cytoplasm, nucleioli differently), Aldehyde Fuchsin(elastin fibers), and Weigert stain (differentiates reticular andcollagenous fibers). A comprehensive list of such stains, theirdescription, and general use is given in R. D. Lillie, “Conn'sBiological Stains”, 8th ed., Williams and Wilkins Company, Baltimore,Md. (1969). Suitable mordants and compositions of the preceding arewell-known to one of skill in the art.

The following, non-limiting list provides exemplary fluorescenthistological stains and their target cells, subcellular compartments, orcellular components if applicable: 4′,6-diamidino-2-phenylindole (DAPI)(nucleic acids), Eosin (alkaline cellular components, cytoplasm),Hoechst 33258 and Hoechst 33342 (two bisbenzimides) (nucleic acids),Propidium Iodide (nucleic acids), Spectrum Orange (nucleic acids),Spectrum Green (nucleic acids), Quinacrine (nucleic acids),Fluorescein-phalloidin (actin fibers), Chromomycin A 3 (nucleic acids),Acriflavine-Feulgen reaction (nucleic acid), Auramine O-Feulgen reaction(nucleic acids), Ethidium Bromide (nucleic acids). Nissl stains(neurons), high affinity DNA fluorophores such as POPO, BOBO, YOYO andTOTO and others, and Green Fluorescent Protein fused to DNA bindingprotein, such as histones, ACMA, Quinacrine and Acridine Orange.

A wide variety of proprietary fluorescent organelle-specific probes areavailable from Molecular Probes (Eugene, Oreg.), which includemitochondria-specific probes (MitoFluor and MitoTracker dyes),endoplasmic reticulum (ER) and Golgi probes (ER-Tracker and variousceramide conjugates), and lysosomal probes (LysoTracker dyes). Theseprobes, as well as many nonproprietary fluorescent histochemicals, areavailable from and extensively described in the Handbook of FluorescentProbes and Research Products 8^(th) Ed. (2001), available from MolecularProbes, Eugene, Oreg.

Each cell containing sample may be co-incubated with appropriatesubstrates for an enzyme that is a cellular component of interest andappropriate reagents that yield colored precipitates at the sites ofenzyme activity. Such enzyme histochemical stains are specific for theparticular target enzyme. Staining with enzyme histochemical stains maybe used to define a subccllular component or a particular type of cell.Alternatively, enzyme histochemical stains may be used diagnostically toquantitate the amount of enzyme activity in cells. A wide variety ofenzymatic substrates and detection assays are known and described in theart, and some selected methods are exemplified below.

Acid phosphatases may be detected through several methods. In the Gomorimethod for acid phophatase, a cell preparation is incubated withglycerophosphate and lead nitrate. The enzyme liberates phosphate, whichcombines with lead to produce lead phosphate, a colorless precipitate.The tissue is then immersed in a solution of ammonium sulfide, whichreacts with lead phosphate to form lead sulfide, a black precipitate.Alternatively, cells may be incubated with a solution comprisingpararosanilin-HCl, sodium nitrite, napthol ASB1 phosphate (substrate),and veronal acetate buffer. This method produces a red precipitate inthe areas of acid phosphatase activity. Owing to their characteristiccontent of acid phosphatase, lysosomes can be distinguished from othercytoplasmic granules and organelles through the use of this assay.

Dehydrogenases may be localized by incubating cells with an appropriatesubstrate for the species of dehydrogenase and tetrazole. The enzymetransfers hydrogen ions from the substrate to tetrazole, reducingtetrazole to formazan, a dark precipitate. For example, NADHdehydrogenase is a component of complex I of the respiratory chain andis localized predominantly to the mitochondria.

Other enzymes for which well-known staining techniques have beendeveloped, and their primary cellular locations or activities, includebut are not limited to the following: ATPases (muscle fibers), succinatedehydrogenases (mitochondria), cytochrome c oxidases (mitochondria),phosphorylases (mitochondria), phosphofructokinases (mitochondria),acetyl cholinesterases (nerve cells), lactases (small intestine),leucine aminopeptidases (liver cells), inyodcnylate deaminases (musclecells), NADH diaphorases (erythrocytes), and sucrases (small intestine).

Immunohistochemistry is among the most sensitive and specifichistochemical techniques. Each histospot may be combined with a labeledbinding composition comprising a specifically binding probe. Variouslabels may be employed, such as fluorophores, or enzymes, which producea product which absorbs light or fluoresces. A wide variety of labelsare known which provide for strong signals in relation to a singlebinding event. Multiple probes used in the staining may be labeled withmore than one distinguishable fluorescent label. These color differencesprovide a way to identify the positions of specific probes. The methodof preparing conjugates of fluorophores and proteins, such asantibodies, is extensively described in the literature and does notrequire exemplification here.

Although there are at least 120,000 commercially available antibodies,the following lists some exemplary primary antibodies known tospecifically bind cellular components and which are presently employedas components in immunohistochemical stains used for research and, inlimited cases, for diagnosis of various diseases. Anti-estrogen receptorantibody (breast cancer), anti-progesterone receptor antibody (breastcancer), anti-p53 antibody (multiple cancers), anti-Her-2/neu antibody(multiple cancers), anti-EGFR antibody (epidermal growth factor,multiple cancers), anti-cathepsin D antibody (breast and other cancers),anti-Bcl-2 antibody (apoptotic cells), anti-E-cadherin antibody,anti-CA125 antibody (ovarian and other cancers), anti-CA15-3 antibody(breast cancer), anti-CA19-9 antibody (colon cancer), anti-c-erbB-2antibody, anti-P-glycoprotein antibody (MDR, multi-drug resistance),anti-CEA antibody (carcinoembryonic antigen), anti-retinoblastomaprotein (Rb) antibody, anti-ras oneoprotein (p21) antibody, anti-Lewis X(also called CD15) antibody, anti-Ki-67 antibody (cellularproliferation), anti-PCNA (multiple cancers) antibody, anti-CD3 antibody(T-cells), anti-CD4 antibody (helper T cells), anti-CD5 antibody (Tcells), anti-CD7 antibody (thymocytes, immature T cells, NK killercells), anti-CD8 antibody (suppressor T cells), anti-CD9/p24 antibody(ALL), anti-CD10 (also called CALLA) antibody (common acute lymphoblasicleukemia), anti-CD11c antibody (Monocytes, granulocytes, AML), anti-CD13antibody (myelomonocytic cells, AML), anti-CD14 antibody (maturemonocytes, granulocytes), anti-CD15 antibody (Hodgkin's disease),anti-CD19 antibody (B cells), anti-CD20 antibody (B cells), anti-CD22antibody (B cells), anti-CD23 antibody (activated B cells, CLL),anti-CD30 antibody (activated T and B cells, Hodgkin's disease),anti-CD31 antibody (angiogenesis marker), anti-CD33 antibody (myeloidcells, AML), anti-CD34 antibody (endothelial stem cells, stromaltumors), anti-CD35 antibody (dendritic cells), anti-CD38 antibody(plasma cells, activated T, B, and myeloid cells), anti-CD41 antibody(platelets, megakaryocytes), anti-LCA/CD45 antibody (leukocyte commonantigen), anti-CD45RO antibody (helper, inducer T cells), anti-CD45RAantibody (B cells), anti-CD39, CD100 antibody, anti-CD95/Fas antibody(apoptosis), anti-CD99 antibody (Ewings Sarcoma marker, MIC2 geneproduct), anti-CD106 antibody (VCAM-I; activated endothelial cells),anti-ubiquitin antibody (Alzheimer's disease), anti-CD71 (transferrinreceptor) antibody, anti-c-myc (oncoprotein and a hapten) antibody,anti-cytokeratins (transferrin receptor) antibody, anti-vimentins(endothelial cells) antibody (B and T cells), anti-HPV proteins (humanpapillomavirus) antibody, anti-kappa light chains antibody (B cell),anti-lambda light chains antibody (B cell), anti-melanosomes (HMB45)antibody (melanoma), anti-prostate specific antigen (PSA) antibody(prostate cancer), anti-S-100 antibody (melanoma, salvary, glial cells),anti-tau antigen antibody (Alzheimer's disease), anti-fibrin antibody(epithelial cells), anti-keratins antibody, anti-cytokeratin antibody(tumor), anti-alpha-catenin (cell membrane), and anti-Tn-antigenantibody (colon carcinoma, adenocarcinomas, and pancreatic cancer).

Fluorophores that may be conjugated to a primary antibody include butare not limited to Fluorescein, Rhodamine, Texas Red, Cy2, Cy3, Cy5,VECTOR Red, ELF™ (Enzyme-Labeled Fluorescence), Cy0, Cy0.5, Cy1, Cy1.5,Cy3, Cy3.5, Cy5, Cy7, FluorX, Calcein, Calcein-AM, CRYPTOFLUOR™'S,Orange (42 kDa), Tangerine (35 kDa), Gold (31 kDa), Red (42 kDa),Crimson (40 kDa), BHMP, BHDMAP, Br-Oregon, Lucifer Yellow, Alexa dyefamily, N-[6-(7-nitrobenz-2-oxa-1,3-diazol-4-yl)aminolcaproyl] (NBD),BODIPY™, boron dipyrromethene difluoride, Oregon Green, MITOTRACKER™Red, DiOC.sub.7 (3), DiIC.sub.18, Phycoerythrin, Phycobiliproteins BPE(240 kDa) RPE (240 kDa) CPC (264 kDa) APC (104 kDa), Spectrum Blue,Spectrum Aqua, Spectrum Green, Spectrum Gold, Spectrum Orange, SpectrumRed, NADH, NADPH, FAD, Infra-Red (IR) Dyes, Cyclic GDP-Ribose (cGDPR),Calcofluor White, Lissamine, Umbelliferone, Tyrosine and Tryptophan. Awide variety of other fluorescent probes are available from and/orextensively described in the Handbook of Fluorescent Probes and ResearchProducts 8^(th) Ed. (2001), available from Molecular Probes, Eugene,Oreg., as well as many other manufacturers.

Further amplification of the signal can be achieved by usingcombinations of specific binding members, such as antibodies andanti-antibodies, where the anti-antibodies bind to a conserved region ofthe target antibody probe, particularly where the antibodies are fromdifferent species. Alternatively specific binding ligand-receptor pairs,such as biotin-streptavidin, may be used, where the primary antibody isconjugated to one member of the pair and the other member is labeledwith a detectable probe. Thus, one effectively builds a sandwich ofbinding members, where the first binding member binds to the cellularcomponent and serves to provide for secondary binding, where thesecondary binding member may or may not include a label, which mayfurther provide for tertiary binding where the tertiary binding memberwill provide a label.

The secondary antibody, avidin, strepavidin or biotin are eachindependently labeled with a detectable moiety, which can be an enzymedirecting a colorimetric reaction of a substrate having a substantiallynon-soluble color reaction product, a fluorescent dye (stain), aluminescent dye or a non-fluorescent dye. Examples concerning each ofthese options are listed below.

In principle, any enzyme that (i) can be conjugated to or bindindirectly to (e.g., via conjugated avidin, strepavidin, biotin,secondary antibody) a primary antibody, and (ii) uses a solublesubstrate to provide an insoluble product (precipitate) could be used.

The enzyme employed can be, for example, alkaline phosphatase,horseradish peroxidase, beta-galactosidase and/or glucose oxidase; andthe substrate can respectively be an alkaline phosphatase, horseradishperoxidase, beta.-galactosidase or glucose oxidase substrate.

Alkaline phosphatase (AP) substrates include, but are not limited to,AP-Blue substrate (blue precipitate, Zymed catalog p. 61); AP-Orangesubstrate (orange, precipitate, Zymed), AP-Red substrate (red, redprecipitate, Zymed), 5-bromo, 4-chloro, 3-indolyphosphate (BCIPsubstrate, turquoise precipitate), 5-bromo, 4-chloro, 3-indolylphosphate/nitroblue tetrazolium/iodonitrotetrazolium (BCIP/INTsubstrate, yellow-brown precipitate, Biomeda), 5-bromo, 4-chloro,3-indolyphosphate/nitroblue tetrazolium (BCIP/NBT substrate,blue/purple), 5-bromo, 4-chloro, 3-indolyl phosphate/nitrobluetetrazolium/iodonitrotetrazolium (BCIP/NBT/INT, brown precipitate, DAKO,Fast Red (Red), Magenta-phos (magenta), Naphthol AS-BI-phosphate(NABP)/Fast Red TR (Red), Naphthol AS-BI-phosphate (NABP)/New Fuchsin(Red), Naphthol AS-MX-phosphate (NAMP)/New Fuchsin (Red), New Fuchsin APsubstrate (red), p-Nitrophenyl phosphate (PNPP, Yellow, water soluble),VECTOR™ Black (black), VECTOR™ Blue (blue), VECTOR™ Red (rod), Vega Red(raspberry red color).

Horseradish Peroxidase (HRP, sometimes abbreviated PO) substratesinclude, but are not limited to, 2,2′ Azino-di-3-ethylbenz-thiazolinesulfonate (ABTS, green, water soluble), aminoethyl carbazole, 3-amino,9-ethylcarbazole AEC (3A9EC, red). Alpha-naphthol pyronin (red),4-chloro-1-naphthol (4CIN, blue, blue-black), 3,3′-diaminobenzidinetctrahydrochloride (DAB, brown), ortho-dianisidine (green), o-phenylenediamine (OPD, brown, water soluble), TACS Blue (blue), TACS Red (red),3,3′,5,5 Tetramethylbenzidine (TMB, green or green/blue), TRUE BLUE™(blue), VECTOR™ VIP (purple), VECTOR™ SG (smoky blue-gray), and ZymedBlue HRP substrate (vivid blue).

Glucose oxidase (GO) substrates, include, but are not limited to,nitroblue tetrazolium (NBT, purple precipitate), tetranitrobluetetrazolium (TNBT, black precipitate),2-(4-iodophenyl)-5-(4-nitorphenyl)-3-phenyltetrazolium chloride (INT,red or orange precipitate), Tetrazolium blue (blue), Nitrotetrazoliumviolet (violet), and3-(4,5-dimethylthiazol-2-yl)-2,5-diphenyltetrazolium bromide (MTT,purple). All tetrazolium substrates require glucose as a co-substrate.The glucose gets oxidized and the tetrazolium salt gets reduced andforms an insoluble formazan which forms the color precipitate.

Beta-galactosidase substrates, include, but are not limited to,5-bromo-4-chloro-3-indoyl beta-D-galactopyranoside (X-gal, blueprecipitate). The precipitates associated with each of the substrateslisted have unique detectable spectral signatures (components).

The enzyme can also be directed at catalyzing a luminescence reaction ofa substrate, such as, but not limited to, luciferase and aequorin,having a substantially non-soluble reaction product capable ofluminescencing or of directing a second reaction of a second substrate,such as but not limited to, lucifcrine and ATP or coelenterazine andCa.sup.++, having a luminescencing product.

The following references, which are incorporated herein provideadditional examples. J. M Elias (1990) Immunohistopathology: A practicalapproach to diagnosis. ASCP Press (American Society of ClinicalPathologists), Chicago; J. F. McGinty, F. E. Bloom (1983) Doubleimmunostaining reveals distinctions among opioid peptidergic neurons inthe medial basal hypothalamus. Brain Res. 278: 145-153; and T. Jowett(1997) Tissue In situ Hybridization: Methods in Animal Development. JohnWiley & Sons, Inc., New York; J Histochem Cytochem 1997 December45(12):1629-1641.

Cellular preparations may be subjected to in-situ hybridization (ISH).In general, a nucleic acid sequence probe is synthesized and labeledwith either a fluorescent probe or one member of a ligand:receptor pair,such as biotin/avidin, labeled with a detectable moiety. Exemplaryprobes and moieties are described in the preceding section. The sequenceprobe is complementary to a target nucleotide sequence in the cell. Eachcell or cellular compartment containing the target nucleotide sequencemay bind the labeled probe. Probes used in the analysis may be eitherDNA or RNA oligonucleotides or polynucleotides and may contain not onlynaturally occurring nucleotides but their analogs such as dioxygenindCTP, biotin dcTP 7-azaguanosine, azidothymidine, inosine, or uridine.Other useful probes include peptide probes and analogues thereof,branched gene DNA, peptidomimetics, peptide nucleic acids, and/orantibodies. Probes should have sufficient complementarity to the targetnucleic acid sequence of interest so that stable and specific bindingoccurs between the target nucleic acid sequence and the probe. Thedegree of homology required for stable hybridization varies with thestringency of the hybridization. Conventional methodologies for ISH,hybridization and probe selection are described in Leitch, et al. InSitu Hybridization: a practical guide, Oxford BIOS ScientificPublishers, Microscopy Handbooks v. 27 (1994); and Sambrook, J.,Fritsch, E. F., Maniatis, T., Molecular Cloning: A Laboratory Manual,Cold Spring Harbor Press (1989).

The present invention is further illustrated by the following examples,which should not he construed as limiting in any way. The contents ofall cited references are hereby expressly incorporated by reference.

EXAMPLE 1 Construction of Tissue Microarrays for a Survival Analysis ofthe Estrogen Receptor (ER) and HER2/neu and for Analysis of NuclearAssociated Beta-Catenin

Tissue microarray design: Paraffin-embedded formalin-fixed specimensfrom 345 cases of node-positive invasive breast carcinoma wereidentified. Areas of invasive carcinoma, away from in situ lesions andnormal epithelium, were identified and three 0.6 cm punch “biopsy” coreswere taken from separate areas. Each core was arrayed into a separaterecipient block, and five-micron thick sections were cut and processedas previously described (Konenen, J. et al., Tissue microarrays forhigh-throughput molecular profiling of tumor specimens, (1987) Nat. Med.4:844-7). Similarly, 310 cases of colon carcinoma were obtained andarrayed, as previously described (Chung, G. G. et al., Clin. Cancer Res.(In Press)).

Immunohistochemistry: Pre-cut paraffin-coated tissue microarray slideswere deparaffinized and antigen-retrieved by pressure-cooking (Katoh, A.K. et al., (1997) Biotech Histochem. F2:291-8). Slides were stained withantibodies to one of three target antigens: monoclonal anti-E.R. (mouse,Dako Corporation, Carpinteria, Calif.), polyclonal anti-HER2/neu(rabbit, Dako Corp.), monoclonal (mouse clone 14, BD Transduction Labs,San Diego Calif.) anti-beta-catenin, or polyclonal rabbitanti-betacatenin. Primaries were incubated overnight at 4° C. Acorresponding goat antimouse or anti-rabbit secondary antibodyconjugated to a horseradish peroxidase decorated dextran-polymerbackbone was then applied for 1 hr (Envision, DAKO Corp.). Targetantigens were either visualized with a visible light chromagen(Diaminobenzidine, DAKO) for visual analysis, or a fluorescent chromagen(Cy-5-tyramide, NEN Life Science Products, Boston, Mass.). Slidesdesignated for automated analysis were counterstained with DAPI forvisualization of nuclei, and either polyclonal rabbit anticytokeratin(Zymed, So. San Francisco, Calif.) or rabbit anti-alpha-catenin (?) todistinguish between tumor cells and stroma as well as to visualize thecell membrane. In many cases, exponentially subtracted images ofhistospots stained with anti-cytokeratin provided an acceptable markerfor the cell membrane due to the sub-membranous coalescence ofcytokeratin in tumor cells. These antibodies were visualized usingeither Cy3- or Alexa 488-conjugated goat anti-mouse or anti-rabbitsecondary antibodies (Amersham, Piscataway, N.J. and Molecular Probes,Eugene, Oreg.). Slides designated for visual inspection werecounterstained with ammonium hydroxide acidified hematoxylin. Manualexamination of microarrays for E.R., HER2neu, and beta-catenin levels tohas been previously described (Snead, D. R. et al., (1993)Histopathology 23:233-8).

Image analysis: Images of microarrays were obtained using a Deltavisionplatform and software (SoftWorx 2.5) (Applied Precision, Issaquah,Wash.), with an attached water-cooled Photometrics series 300 camerathrough a 10× Nikon Super-Fluor lens on a TE200 inverted fluorescentmicroscope with automated X,Y,Z stage movement. Low power images ofmicroarrays were stitched together using multiple (˜1500) low resolutionimages of the microarray (64×64 pixel). These images were analyzed bysoftware algorithms described herein to determine the location of each.Subsequently, monochromatic, high resolution (1024×1024 pixel) imageswere obtained of each, both in the plane of focus and 8 microns belowit. Image pairs for each fluorescent dye were obtained. Images wereanalyzed using additional algorithms as follows, in brief. Regions ofinterest (tumor) were identified using a mask derived from aubiquitously-expressed epithelial-specific antigen (either cytokeratinor alpha-catenin). Images of fluorescently-tagged membrane and nuclearcompartments were exponentially subtracted until a set amount of imageintensity remained. Images were then combined so that there was minimaloverlap of signal from one compartment to the next. Pixels in which asignificant degree of overlap was present were negated from furtheranalysis. The pixel intensity of exponentially subtracted images of thetarget antigen were assigned to one of three compartments: nuclear,membrane, or non-nuclear non-membrane (cytoplasm). Target intensitieswere analyzed as described below. For E.R. only nuclear-localized signalwas used, for HER2/neu only membrane-localized signal was analyzed. Forbeta-catenin total signal, the ratio of nuclear to membrane signal, andthe ratio of nuclear to total signal was analyzed.

Data analysis: staining scores from the breast cancers represent theaveraged (for ER) or maximized (for HER2/neu) results from two scorablehistospots. Subsequent studies revealed that analysis of a singlehistospot could provide significant statistical power to judge outcomes,so that staining scores from the colon cancer array represent the resultof only one histospot. Overall survival analysis was assessed usingKaplan-Meier analysis and the Mantel-Cox log rank score for assessingstatistical significance. Relative risk was assessed using theunivariate Coxproportional hazards model. Analyses were performed usingStatview 5.0.1 (SAS Institute, Cary N.C.).

1. A computer implemented method of localizing a biomarker within acell, comprising: identifying portions of a first image of the cell thatcorresponds to a first defined area; identifying portions of a secondimage of the cell that corresponds to at least one biomarker;determining portions of the second image that lie within the portions ofthe first image to identify whether the biomarker is localized withinthe defined area.
 2. The method of claim 1, wherein the portions of thefirst image and portions of the second image comprise pixels.
 3. Themethod of claim 1, wherein the portions of the first image comprisepixels having an intensity associated with the first defined area of thecell.
 4. The method of claim 3, further comprising determining theintensity associated with the first defined area of the cell.
 5. Themethod of claim 2, wherein the portions of the second image comprisepixels having an intensity corresponding to the at least one biomarker.6. The method of claim 5, further comprising determining the intensityassociated with the at least one biomarker.
 7. The method of claim 1,further comprising reducing representation of out-of-focus elements inthe first image.
 8. The method of claim 7, wherein reducingrepresentation comprises manipulating image pixel intensities of thefirst image based on image pixel intensities of a third image featuringa different depth of focus.
 9. The method of claim 1, wherein thedefined area is selected from the group consisting of the cell nucleus,cytoplasm, nuclear membrane, cellular membrane, mitochondria,endoplasmic reticulum, peroxisome and lysosome.
 10. The method of claim1, wherein the cellular component is selected from the group consistingof a protein, peptide, nucleic acid, lipid or carbohydrate.
 11. A methodof analyzing a cell containing sample, comprising: obtaining a firstimage of a cell with a first stain that is selective for a first definedarea within the cell and a second stain that is selective for at leastone biomarker; determining an intensity value for the first stain at aplurality of pixel locations in the first image; based on the intensityvalues, determining pixel locations in the first image that correspondto the first defined area within the cell and assigning those pixellocations to the first defined area; obtaining a second image anddetermining an intensity value for the second stain at a plurality ofpixel locations in the second image; and comparing the first and secondimages to identify pixel locations in the second image that are withinthe first cellular compartment.
 12. The method of claim 11, wherein thedefined area is selected from the group consisting of the cell nucleus,cytoplasm, nuclear membrane, cellular membrane, mitochondria,endoplasmic reticulum, peroxisome and lysosome.
 13. The method of claim11, wherein the cellular component is selected from the group consistingof a protein, peptide, nucleic acid, lipid or carbohydrate.
 14. Themethod of claim 11, wherein the cell is contacted with a third stainthat is selective for a second defined area within the cell, the methodfurther comprising: in a third image of the distribution of the thirdstain in the cell, determining an intensity value for each pixellocation of a plurality of pixel locations in the third image; based onthe intensity values, determining which pixel locations in the thirdimage correspond to the second defined area of the cell and assigningthose pixel locations to the second defined area; and identifying whichof the pixel locations in the second image are within the second definedarea.
 15. The method of claim 14, further comprising assigning pixellocations not assigned to the first or second defined areas to a thirddefined area, and identifying which of the pixel locations in the secondimage are within the third defined area.
 16. The method of claim 15,wherein the first, second, and third defined areas are selected from thegroup consisting of: a nucleus, cytoplasm, nuclear membrane, cellularmembrane, mitochondria, endoplasmic reticulum, peroxisome and lysosome.17. The method of claim 14, wherein the cell is contacted with a fourthstain that is selective for a defined area in the cell, and at least onepixelated image of the distribution of the fourth stain is acquired, themethod further comprising reading a third intensity value for each of aplurality of pixels in the image of the fourth stain distribution;determining a threshold intensity value from the third intensity values;comparing the third intensity value for each of the plurality of pixelsto the threshold intensity; and assigning pixel locations to a maskbased on the threshold intensity value.
 18. The method of claim 17,wherein the pixel locations in the plurality of pixels in the image ofthe first stain distribution are the pixel locations in the mask set.19. The method of claim 18, wherein the pixel locations assigned to themask comprise the location of pixels having third intensity values equalto or greater than the threshold intensity value. 20-32. (canceled) 33.A method of analyzing a plurality of spots, comprising: locating thespots; defining a reference point for the located spot; connecting thereference point of the located spot to the reference point of a set ofnearest neighbor spots with a corresponding line segment; andidentifying points of intersection between line segments used to connectspots to each other. 34-38. (canceled)